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Abstract 



We continue development of the theory of Markov systems initiated in 
[34]. In this paper, we introduce fundamental Markov systems associated 
with random dynamical systems and show that the proof of the unique- 
ness and empiricalness of the stationary initial distribution of the random 
dynamical system reduces to that for the fundamental Markov system 
associated with it. The stability criteria for the latter are much clearer. 
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In [34], the author initiated the study of a general concept of a Markov system. 
This was motivated by a desire to have a scientifically consistent unifying math- 
ematical structure which would cover finite Markov chains [10], (/-measures [25] 
and iterated function systems with place- dependent probabilities [1], [13]. 

The purpose of this note is to show that the structure of a Markov system arises 
naturally (possibly unavoidably) in the study of random dynamical systems. 

1 Random dynamical systems 

Let {K,d) be a complete separable metric space and E a countable set. For 
each e S E let a Borel measurable map w e : K — ► K and a Borel measurable 
probability function p e : K — ► [0, 1] be given, i.e. 



We call the family T> := (K,w e ,p e ) e( zE a random dynamical system. A survey 
on random dynamical systems can be found e.g. in [23]. If a reader doesn't see, 
how the definition of random dynamical system in [23] relates to that in this 
paper, it is explained in [24]. 




for all x S K. 
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With the random dynamical system T> is associated a Markov operator U acting 
on all bounded Borel measurable functions / by 

Uf ■= ^Pef OW e 

e£E 

and its adjoint operator acting on the set of Borel probability measures v by 

U*u{f) := J U(f) dv. 

A measure \i is called invariant with respect to the random dynamical systems 
if and only if 

U*H = p. 

Let E + := {(ci, 02, ...) : <7j G E, i G N} endowed with the product topology of 
discreet topologies and 5 : S + — ► E + be the left shift map. For xeK, let P x 
be the Borel probability measure on S + given by 

Px(i[ei,...,e„]) -~p ei (x)p e2 ow ei (x)...p en ow en _ 1 o ... o w ei (x), 

for every cylinder set i[e\, ...,e„] := {a G E + : ui = e\, ...,a n = e n }, which is 
associated with the Markov process generated by the random dynamical system 
with the Dirac initial distribution S x . 

Remark 1 Note that each map w e needs to be defined only on a subset of K 
where its probability function p e is greater than zero. In this case, one obtains 
a random dynamical system on K by extending the maps on the whole space 
arbitrarily. 

1.1 Historical context 

First, I will give some historical roots of the theory, and then, I will list some 
works which in my view form the historical context of this paper. 

It was not clear until the beginning of the twentieth century whether the in- 
dependence of random variables is a necessary condition for the low of large 
numbers and the central limit theorem to hold. The breakthrough was a work 
by A. A. Markov [26] in which he has extended the low of large numbers to 
dependent random variables (he also extended the central limit theorem to such 
processes, see [11] for a nice account on Markov's work and life). Markov re- 
stricted himself to processes where each random variable depends only on the 
previous one. Such a process, in case of a discreet state space, is generated 
by a transition matrix (or a directed graph with probability weights) and an 
initial distribution. These processes, now known as Markov chains, found many 
applications. 
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From the work of Markov naturally arises the question whether the low of large 
numbers and other limit theorems hold true for a more general class of depen- 
dent processes. The truly next class of processes can be only those where the 
dependence of a random variable on the past is not restricted to any number of 
previous variables. The study of such processes was initiated (and motivated by 
applications) by 0. Onicescu and G. Mihoc [27]. Remarkably, they found a way 
of constructing such processes without giving infinitely many rules of depen- 
dence of a random variable on the values of all previous, which is of course very 
important for applications. They called such processes les chaines a liaisons 
completes. Their work gave rise to the theory of dependence with compete con- 
nections, in which many limit theorems have been proved [17]. What I call in 
this paper random dynamical system can be seen as a special case of this theory. 
However, it must be said that every stationary process with values in E can 
already be generated by such a random dynamical system. Let's illustrate the 
dependence in the notation of this paper. If maps w e are contractions on a 
complete metric space, the past is coded by the maps to a point in topological 
space K and the probability of a value from E of the next random variable is 
then obtained as a function evaluated at that point, 



P(Xi = ei\X = e ,X_i = e_i,...) = p ei ( hm w eo o w e _ 1 o ... o w e _ n (x ) 



One could argue that the development so far was guided mainly by the inter- 
nal mathematical logic. However, the main reason for the development of the 
mathematical language still is the striving of Homo Sapiens for a better descrip- 
tion of the world outside (even if many modern mathematical craftsmen have 
no interest in science at all). The random dynamical systems such as in this 
paper arose naturally also as learning models [20], [9]. Consider an intelligent 
object, e.g. a rat. Its state of "intelligence" x is assumed to take some value 
from [0, 1] C R. The object is "asked some questions" and its responses are 
measured as '0' ("wrong") or T' ("right"). If it gives an answer '0', its "intel- 
ligence" is moved to Wo(x) :— l/2x. Otherwise, its "intelligence" is moved to 
W\(x) := l/2a; + 1/2. It is natural to assume that a response '0' can happen 
with some probability po(x) (depending on the current state of "intelligence") 
and a response T' with probability pi(x) := 1 — po(x). One assumes that the 
function p does not change with the time (if it did, it's likely that it would 
have some time average, which only would matter [3]). Furthermore, probabil- 
ity functions po and p\ easily can be obtained empirically if sufficiently many 
measurements are made. Many seminal works on Markov processes generated 
by random dynamical systems have been motivated by learning models. 

The striving for a classification of stationary random processes brought attention 
to such systems once again. After Ornstein [28] had shown that Kolmogorov- 
Sinai entropy is a complete invariant on Bernoulli shifts and not complete on 
Kolmogorov automorphisms [29], arose an interest in measure preserving trans- 




fer all (..., e_i, eo, ei, ...) <E E and xq e K . 
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formations which are Kolmogorov and not Bernoulli. Of course, a natural candi- 
date for that is a strongly mixing process on a finite alphabet with infinite mem- 
ory, which is known to be generated by these kind of random dynamical systems. 
It was M. Keane [21] who drew attention of ergodic theory people to such sys- 
tems. He considered a special case where the maps of the system can be obtained 
as inverse branches of a (expanding) homomorphism S of a compact metric space 
K (e.g. K := _E Nu0 and S is shift map, then w e (..., y~i, yo) '■= (■■■, V-i,yo, e) for 
all (..., y-i, yo) G K). In this case, the probabilities for these maps can be given 
in terms of a function g : K — ► [0, 1] (p e (—, y~i, yo) ■= #(•••, 2/0, e) for all 
(..., y_i, yo) 6 K) with the property that J2 ye s-i({ x }) div) ~ 1 f° r an x e ^0- 
Then the stationary process also is invariant with respect to homomorphism S. 
He called the stationary process in this case a g-measure. Much of the progress 
on the subject has been made from the study of this special case. Of course, the 
one-sided symbolic space is more than just an example. However, the restriction 
to this case clearly encourages an algebraic rather than a geometric approach, 
which could be one of the reasons why the structure of the fundamental Markov 
system, which is going to be introduced in this paper, can not have been seen 
before. 

A careful reader probably has noticed that in all of the examples so far the maps 
of the system have been contractive. It was shown by J. Hutchinson [16] that 
a family of contractive maps on a compact metric space has a unique invariant 
subset. These subsets are like footprints of this kind of "animals". They are 
often very irregular on every scale. (This must be very fascinating already for 
itself because there are even people who study these footprints without actually 
any interest in any "animal".) Advances in computational technology allowed 
to produce beautiful colorful pictures of such sets. Naturally, arose the idea 
of storing images with such systems [14]. Moreover, contractive maps allow to 
code strings of symbols such as '0' and T' to a point in the state space. This 
can be useful for data compression [4]. This all attracted a new interest to the 
random dynamical systems. The study of such systems for these purposes was 
continued mainly by M. F. Barnsley et al. [1], [2]. They called them iterated 
function systems with place dependent probabilities. Much of the literature on 
the subject is now available also under this name. 

Recently, such system have been suggested also as a natural model for a quantum 
measurement process [31]. Let K be & state space of a quantum system and 
E a set of possible outputs of a measurement apparatus. When the quantum 
system being in a state x £ K interacts with the measurement apparatus and 
we observe e € E, it is known that this usually results in the system moving to a 
new state y £ K. Hence, it is natural to assume that there is a map w e : K — > K 
associated with each output e G E. Moreover, it is natural to assume that e 
could be observed with some probability p e (x) in this experiment. This all 
also can be formulated using the conventional language of Hilbert space and 
projection operators [31]. The reader probably has noticed the analogy with 
the setup of the "learning model" above. It probably is an indication on the 
universality of the approach. In fact, studying a system by asking it some 
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"question" is a very natural approach. 

I hope I have convinced the reader that the structure which is being studied 
in this paper is not one of the toys designed by some mathematicians to keep 
them busy, but one which naturally has been crystalizing in the modern science. 
It clearly needs to be properly integrated into the body of mathematics which 
describes deterministic, random and quantum paradigms. Now, I will list some 
important contributions which were made for the purpose of understanding such 
systems with respect to their stability and ergodic properties which clearly form 
the historical context of this paper. 

W. Doeblin and R. Fortet [12] (1937) gave fairly weak condition on strictly 
positive probability functions on a compact metric space which insures that 
the system with contractive maps has a unique (attractive) stationary initial 
distribution. In particular, this condition is satisfied if the probabilities have a 
summable variation (Dini-continuous). 

L. Breiman [9] (1960) proved the strong low of large numbers for Markov opera- 
tors with the Feller property which posses a unique stationary initial distribution 
on a compact Hausdorf space (see a sharper result in [37]). 

R. Isaac [18] (1962) introduced the average contractiveness condition, which 
insures the uniqueness of the stationary state (he proved it on a compact metric 
space with strictly positive probability functions satisfying Lipschitz continuity) . 

F. Ledrappier [25] (1974) identified the g-measures as projections of some equi- 
librium states defined by variational principle with respect to the potential log g 
seen as a function on E z (see [38] for the explanation in the general case). 
Furthermore, he showed that the natural extension of the g-measure is weakly 
Bernoulli if the (/-function is strictly positive and Dini-continuous. P. Walters 
[33] (1975) extended the result to such g on a subshift of finite type (Markov 
system). 

T. Kaijser [22] (1981) introduced a local contractiveness on average condition 
in the general setup of random systems with complete connections, which also 
can imply the uniqueness of the stationary state. He called his systems weakly 
distance diminishing random systems with complete connections. 

H. Berbee [5] (1987) showed the uniqueness and the very weak Bernoulli prop- 
erty of the (/-measure for strictly positive (/-functions on a full shift satisfying a 
continuity condition which is weaker than the Dini-continuity. After that, many 
other works have been devoted to the weakening of the algebraic expression forc- 
ing a continuity of the g-function, but mostly only for the proof of uniqueness of 
the stationary state (see 0. Stenflo [30] (2003) and N. Berger, Ch. Hoffman, V. 
Sidoravicius [6] (2005) and the references there). (It must be pointed out that 
the continuity of the probability functions is not fundamental for the stability 
of such systems (e.g. Example 2)). 

J. H. Elton [13] (1987) recognized the importance of the relation for x,y G K 
given by the equivalence of measures P x and P y for the proof of the ergodic 
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theorem for such systems (though, he still assumed the uniqueness of the sta- 
tionary state, which was shown later [35] to be not necessary). He proved that 
this equivalence relation holds true for all x, y G K if all probability functions 
are bounded away form zero, Dini-continuous and the system satisfies a contrac- 
tiveness on average condition (in the language of this paper, the fundamental 
Markov system associated with such a random dynamical system has a single 
vertex set). 

M. F. Barnsley, S. G. Demko, J. H. Elton and J. S. Geronimo [1] [2] (1989) 
made, in my view, two important contributions to understanding the conditions 
for the stability of such systems. They showed that the condition of the strict 
positivity of the probability functions can be weakened (is not fundamental). 
Secondly, they found implicitly a way of reduction of the multiplicative aver- 
age contractiveness condition to the additive average contractiveness condition, 
though they did not accomplish it completely (see [34] for details). 

I. Werner [35] (2005) showed that the condition of equivalence of measures P x 
and P y for all x and y in the same vertex set of a Markov system (see next 
section), which is continuous, irreducible and contractive, is sufficient for the 
uniqueness of the stationary initial distribution. For example (the example was 
given by using some ideas of A. Johansson and A. Oberg [19]), this condition is 
satisfied if the probability functions have a square summable variation on each 
vertex set and are bounded away from zero, which is more general than Elton's 
[13] example. 

This list is far from being complete. There are many other works, which a 
reader can easily find under code names g-measures, iterated functions systems 
with place- dependent probabilities, random systems with complete connections, 
random dynamical systems and Markov systems. A reader interested in the 
study of general Markov operators is referred to [32]. 



1.2 Markov systems 

Now, let us consider a special random dynamical system which we call a Markov 
system [34]. 

Let Ki,K 2 , Kjy be a partition of a metric space K into non-empty Borel sub- 
sets (we do not exclude the case A — 1). Furthermore, for each i e {1, 2, N}, 
let 

Wii,Wi2, ■■■ 1 w iLl : Ki — > K 

be a family of Borel measurable maps such that for each j € {1, 2, Li} there 
exists n € {1,2,..., A} such that Wij {Ki) C K n (Fig. 1). Finally, for each 
i e {1,2,..., A}, let 

Pn,Pi2, ■■-,PiL % ■ Ki — > R + 

be a family of positive Borel measurable probability functions (associated with 
the maps), i.e. pij > for all j and Y^j=i Pij( x ) = 1 f° r an x ^ Ki. 
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N = 3 




Ki w e5 K 3 

Fig. 1. A Markov system. 

Definition 1 We call V := {1, N} the set of vertices and the subsets K\, Kn 
are called the vertex sets. Further, we call 

E := {{i,rii) : i G {1, iV},^ £ {1, ...,£;}} 

ifte sei o/ ed^es and we use the following notations: 

Pe ■= Pin and w e := w in for e := (i, n) G 

Each edge is provided with a direction (an arrow) by marking an initial vertex 
through the map 

i:E — ► V 
(j, n) i — ► j. 

The terminal vertex t(j,n) G 7 of an edge (j,n) G S is determined by the 
corresponding map through 

t((j,n)) := k :<=> w jn (Kj) C K k . 

We call the quadruple G := (V, .E, a directed (multi)graph or digraph. A 
sequence (finite or infinite) (..., e_i, eo, ei, ...) of edges which corresponds to a 
walk along the arrows of the digraph (i.e. t(ek) — i(ek+i)) is called a path. 

Definition 2 We call the family A4 := (-fQ( e ), w e ,p e ) e ^ E a (finite) Markov 
system. 

The definition can be easily generalized to the infinite case. 

The Markov system defines a random dynamical system on K by extending 
the probability functions p e |if i(c) on the whole space by zero and the maps 
arbitrarily, as in Remark 1. 
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Definition 3 We call a Markov system irreducible or aperiodic if and only if 
its directed graph is irreducible or aperiodic respectively. 

Definition 4 (CMS) We call Markov system M contractive with an average 
contracting rate < a < 1 if and only if it satisfies the following condition of 
contractiveness on average: 

Pe(x)d(w e (x),w e (y)) < ad(x,y) for all x,y G Ki and i G {1, ...,N} (1) 

(it is understood here that p e 's are extended on the whole space by zero and 
to e 's arbitrarily). This condition was discovered by Richard Isaac in 1962 for 
the case N = 1 [18]. 

It was shown in [35] that an irreducible contractive Markov system M with 
uniformly continuous probabilities p e \K i(e) > has a unique invariant Borel 
probability measure if P x <C P y for all x,y € -fQ( e ), e & E, and the subsets Ki 
form an open partition of K (this was shown in [35] for some locally compact 
spaces, but it holds also on complete separable spaces, as contractive Ai also 
posses invariant measures on such spaces [15]). 

1.3 Fundamental Markov systems 

Now, we intend to show that with every random dynamical system V is asso- 
ciated an equivalent Markov system M' := {KL^, w' e ,p' e ) ee E< (not necessarily 
finite) such that P' x -C Py for all x,y E K'i( e \, e S E' , and each is the 

largest with such property, where P' x are the probability measures on the code 
space of M.' . 

The construction of M! goes as follows. Define an equivalence relation between 

x, y e K by 

x ~ y :^ P x O 

where P x P y means P x is absolutely continuous with respect to P y and P y 
is absolutely continuous with respect to P x . Let 

l+J K'i = K 

iev 

be the partition of K into the equivalence classes. Then, for every e & E and 
x,y e i £ V, 

p e (a;) = ^ p e (y) = 0. 
Hence, for every e € E and isV", 

either p e |^ = or p e |^ > 0. (2) 

Furthermore, holds the following. 
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Proposition 1 For every ee£ and i £ V with p e \K' i > 0, there exists j E V 
such that We^K'i) C K'j. 

Proof. Let x,y E if-. Observe that 

fx(i[e,cri, ...,cr„]) =p e (a;)P ll , e ( a; )(i[o-i, ...,cr„]) 
for every cylinder set i[<Ji, ...,a n ]. Hence, 

P X {S-\B) ni [e]) 



P Wll (x){B) = 



Pe(x) 



for every Borel -B C S + . Since the analogous formula holds true also for P We ( y ), 
we conclude that w e (x) ~ w e (y). Thus, there exists j £ V such that w e (if 4 ') C 
Kj. ■ □ 

By (2) and Proposition 1, we can define a Markov systems associated with V. 
Definition 5 Let 

El := {(i, e) : p e \ K , > 0, e E E} for all i E V' 

and 



Ef := (J El. 



iev 

For every (i,e) E E' set p' {ie) := p c Ik>, w{i , e) := w e |^, e)) = i and 
f((i,e)) = j where w e (^') C X/. Then G" := (V, £"*, z', t') is a directed 
graph and we call M 1 := (if?^, u> e ,p e ) e e£' the fundamental Markov systems 
associated with the random dynamical system V. 

Now, we need to show that the vertex sets of the fundamental Markov system M! 
associated with V are measurable. Otherwise, possible Banach-Tarski effects 
might make our construction scientifically irrelevant. For that, we need to make 
clear the constructive nature of the equivalence relation which defines the vertex 
sets. 




p*(iki,...<T„]) 

Py(i[ai,...a n ]) ' 



P y (i[ai, ...a n }) > 

0, P*(l[<7l,...<7„]) =0 

oo, Px(i[<7i, •••c„]) > and P y (i[cri, ...a n }) = 



and 



Pffe^fr P x ( 1 [a 1 ,...<7„])>0 
Y n {a):=\ 0, P y { 1 [a 1 ,...a n }) = 

oo, PydWi, -o-„]) > and Pcd^i, ...a n ]) = 
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for all a £ S + . Define 



£(x,y) := lim sup sup (X„ > M) + lim sup sup P y (Y n > M). 

M— too nGN M^oo nGN 

Observe that each x i — ► P x (i[ci, ■•■c n ]) is a Borel measurable function. There- 
fore, each x i — ► P x (X n > M) is a Borel measurable function. Hence, a; i — ► 
£(x, y) is a Borel measurable function for all y £ K. By the symmetry, also 
y i — ► £(x, y) is a Borel measurable function for all x £ K . 

Lemma 1 For all x,y £ K , 

£(x, y) = if and only if P x <C^> P y . 

Proof. Let x,y £ K . Let A n be the finite cr-algebra on S + generated by the 
cylinders i[o\, ...a n ]. Now, observe that, for all m < 



/X n (IPy — P X (C n ) — P X {Cm) — \ X m dPy. 



(3) 



Hence, (X n ,A n )neN is a P y -martingale. Analogously, (Y n ,A n )neN is a Pr 
martingale. Moreover, by (3), 



P x (X n >M)= J X 



dP 



{X n >M} 

and analogously 

dP, 

{Y„>M} 

Hence, 



P y (Y n > M) = J Y n 



£(x,y) = lim sup sup / X n dP y + lim sup sup / Y n dP x . 

M^oo n£N J M^oo n£N J 



{x„>m} {y„>M} 



Therefore, £(x,y) = if and only if X n and Y n are uniformly integrable mar- 
tingales. Hence, the condition £(x,y) — implies that there exists X £ £ 1 (P !/ ) 
and Y £ £ 1 (P C ) such that X n — ► X and y„ — > Y both in £ 1 sense, and 
E Py {X\A m ) = X rn Py-a.e. and E Px (Y\A m ) = Y m P x -&.e. for all m. Then, by 
(3), 

J X dP y = j X m dP y = P x (C m ) for all C m £ A m . 



c„ 



Hence, the Borel probability measures XP y and P x agree on all cylinder subsets 
of E + , and therefore, are equal. Analogously, YP X = P y . Thus, P x <C^> P y . 

Conversely, P x P y implies that X n and Y n are uniformly integrable [7], i.e. 
£(x,y)=0. ' ' □ 
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Remark 2 Note that it is not obvious from the definition of £ that the relation 
£(x,y) = is transitive. 

Proposition 2 (i) The vertex sets K[, i G V' , are Borel measurable, 
(ii) Consider all probability functions p' e \K' { ) , e S E' , to be extended on K by 
zero and all maps w' e \K> , e G E' , to be extended on K arbitrarily. Let U' be 
the Markov operator associated with the Markov system M! . Then U' — U , i.e. 
M 1 is an equivalent random dynamical system to V. 

Proof, (i) Let i G V . Fix y G K[ and set f(x) := £{x,y) for all x G K. Then, 
by Lemma 1, K[ = / _1 ({0}). Hence, as / is Borel measurable, K[ is Borel 
measurable. 

(ii) Let g be a bounded Borel measurable function on K and x G K. Then there 
exists a unique i G V such that x G K[. Hence, by the definition of M, 

U'g{x) = ^p' e (x)gow' e (x) = ^2p' e (x)gow' e (x) = ^2p e (x)gow e (x) 

eeE' eEE't eeE 

= Ug{x). 

□ 

Example 1 Suppose the random dynamical system T> is given by the con- 
tractive Markov system A4 such that the vertex sets K±, ...,Kn form an open 
partition of K and the probability functions Pe\K i{c) are bounded away from 
zero and have a square summable variation, i.e. X)«eN 2 ( a ") < °°> where <j> 
is the maximum of modules of uniform continuity of functions Pe|if i(e) , e G E. 
Then, by Lemma 2 in [35], P x <§; P y for all x,y G K iy i = 1, ...,iV (note that 
the openness of the partition was required in [35] only to insure that M. has 
an invariant measure (Feller property)). Therefore, the fundamental Markov 
system associated with M is M itself. 

Example 2 Let T>i := ([0, 1], w e ,p e ) e =o.i be the random dynamical system 
where wq(x) — x/3, w\(x) = x/3 + 1/3 for all x G [0, 1], 

, f 0, < x < I 
MX) = { b, l<x<l ' 

with < b < 1, and p\ = 1 — po- 

Claim 1 The vertex sets of the fundamental Markov system associated with T>2 
are K := [0,1/9], K x := (1/9,1/3] and K 2 := (1/3,1]. 

Proof. First, observe that partition Kq U K\ U K 2 makes a Markov system with 
constant probabilities from T>2, where the transition matrix associated with it 
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A 2 := 
Therefore, 

Pe n ( w e n -i ° — °W ei x) = p erl (w en _ 1 o ... o w ei y) 

for all x,y £ Ki, i = 0, 1, 2 and ei, e n e {0, 1}. Thus 

■P* = P y for all i,|/e Kq,Ki,K 2 . 

It remains to show that Kq, Ki and if 2 are the largest with the property that 
P x «» P v for all x,y € Ki, i = 0,1,2. Let x G K and y K . Then 
P x (i[0]) = 0, but Pj,(i[0]) = 6 > 0. Now, let x G K x and y £ ATi. If y G if , 
then P y (i[0}) = 0, but P K (i[0]) = b. Otherwise, if y G K 2 , Px(i[00]) = 0, but 
Py(i[00]) = b. The claim follows. □ 

Now, we can apply Theorem 2 in [36] to an equivalent fundamental Markov 
system on a disconnected set, the vertex sets of which are K := [0, 1/9], K\ := 
[2/9,1/3] and K 2 := [2/3,1]. (Note that there is a missprint in [36] on page 
471. It should be A := D~ 1 A t D.) By Theorem 2 in [36], V 2 has a unique 
invariant Borel probability measure /i 2 with h 2 {Kq) = b 2 /(l + b + b 2 ), fi 2 (Ki) = 
6/(1 + b + b 2 ) and fi 2 (K 2 ) = 1/(1 + b + b 2 ) and the Markov chain associated 
with V 2 is geometrically ergodic with a relative rate of convergence in Monge- 
Kantorovich metric less or equal to max{l/3, 6} 1 / 2 . 



If we replace po with 

Po(x) 



0, < x < ± 
b, < x < 1 



the same argumentation as in the proof of Claim 1 shows that the fundamental 
Markov system associated with modified T> 2 has four vertex sets Kq := [0, 1/27], 
Ki := (1/27, 1/9], K 2 := (1/9, 1/3] and K 3 := (1/3, 1] with constant probabili- 
ties on them given by the transition Matrix 



A, 



/ 1 \ 

b 1-6 

6 1-6 

\ 6 1 - 6 J 



Analogously, applying Theorem 2 in [36] gives that the modified V 2 has an 
attractive invariant probability measure jj,3 with the weights on vertex sets 
Ms(^o) - b 3 /(l + b + b 2 + 6 3 ), M3 (i^i) - b 2 /(l + b + b 2 + 6 3 ), p 3 (K 2 ) = 
6/(1 + 6 + 6 2 + 6 3 ) and ^3(^4) = 1/(1 + 6 + 6 2 + 6 3 ). (Somewhat surpris- 
ingly, the estimation of the relative rate of convergence by Theorem 2 in [36] 
remains the same (the absolute value of eigenvalues of such transition matrices 
remains the same and equals 6) . Note that in case of po = 6 the rate of con- 
vergence to the stationary state in the Monge-Kantorovich metric is not greater 
than 1/3.) 
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Example 3 Let P3 := ([0, 1], w e ,Pe)e=o.i be the random dynamical system 
where Wo(x) — x/3, w\(x) = x/3 + 1/3 for all x G [0, 1], 

/ x J 3, < x < \ 

Mx) = \ I l<x<i 

and pi = 1 -p - As p CT2 (w ai x) = p a2 (w ai x) for all x,y G [0, 1], <7i,er 2 G {0, 1} 
and Po > 0, follows << P y for all i,t/£ [0, 1]. Thus the fundamental Markov 
system associated with T>z has the single vertex set. 

Example 4 Let V4 := (R, w e ,p e ) e =o,i be the random dynamical system where 
u; (or) = x/2, wi(x) = x/2 +1/2 for all x G E, 



Po(z) = { r 

I 3' 



x G E \ 1 



and pi = 1 — po- If a; is rational, then all images of x under wq and w\ are 
rational. Therefore, P x is the Bernoulli measure generated with probabilities 
{1/4,3/4}. Analogously, for irrational y, P y is the Bernoulli measure generated 
with probabilities {1/3, 2/3}. Since P x and P y can not be absolutely continuous 
(absolutely continuous ergodic measures are equal), the fundamental Markov 
system associated with V4 has two vertex sets Kq := Q and K\ := E \ Q. 



Having obtained the well defined fundamental Markov system associated with 
V, we can define a Borel probability measure P' x on £'+ := {(ai,a 2 , ■■■) ■ <Ji G 
E' \/i G N} (provided with the product topology of discreet topologies) by 

P x ( 1 [e 1 ,...,e n }) :=p' ei (x)...p' en (w' en _ 1 o...ow' ei x) 

for all cylinder sets i[ei, ...,e„] C for every x G K. 

Now, we need to establish a relation between measures P' x and P x . 

Since for each e' G E' , Wg/|/f' ( , is a restriction of a unique w e , we can define 
a map tp : E' — > E by ip(e')=e. This gives a Borel-Borel-measurable map 
* : £'+ — ► £+ by (*(<r))i := V>(oi) for alH G N and for all a G £'+. 

Lemma 2 For every x £ K, P x — ~$>{P' X ). 



Proof. Let i[ei, ...,e„] C S + be a cylinder set. Then, by the definition of 'J, we 
can write ^E' _1 (i[ei, e„]) as a disjoint union of some cylinder sets i[e[, e^J C 
£'+, i.e. 

*- 1 ( 1 [e 1 ,...,e„])= |+J i[ei,...,e;]. 

K>--->e^),V'(e-)=e» 

Therefore, 

*(i^)( 1 [e 1 ,...,e n ])= X! Pei^-^o^o- o^W- 

(ei,...,e^),'i/'(e<)=ej 



13 



Now, observe that, by the definition of p' e , 

e',V'(e / )=e 

This implies that 

*(-Px)(i[ei,-,e„]) =p ei (x)...p en ow £nl o...ow ei (x) = P^ei, e„]). 
Thus, the claim follows. □ 

Proposition 3 For all x,y G K , 

P X <^Py O P' x <^P' y . 

Proof. ' <=' Let A C S + Borel measurable such that P a (A) = 0. Then, by 
Lemma 2, P y ('i>^ 1 (A)) = 0. Hence, by Lemma 2 and the hypothesis, P X (A) = 
P^-^A)) = 0. Thus P x « P y . 

' First, observe that, by the construction of M' , for every cylinder set 
i[e[, e' n ] C S' + with P y (i[e[, ...,€'„]) > 0, there exists a unique cylinder set 
i[ei, ...,e„] C S+ such that P^(i[ei, ...,<]) = P y (i[ei, ...,e„]) and*(i[ei, ...,<]) =i 
[ei, ...,e„]. Hence 

P y ( 1 [e' 1 ,...,e' n })=P v m 1 [e'i,..,e , n })). (4) 

Now, let B C £' Borel measurable such that P^(-B) = 0. Let e > 0. By the 
hypothesis, there exists S > such that 

Py(C) < 6 P X (C) < e (5) 

for all Borel measurable C C S + . By the Borel regularity of P' x , there exists a 
countable family of cylinder sets Ck C k G N, such that i? C lj fc Cfc and 
Py({J k Ck) < S. Since we can write every finite union \J™ Ck as a disjoint union 
of cylinder sets U^ m Ck , 

oo 

Y, P vV)=P' y {}C k )<5. 
fe=l fc 

Hence, by (4) and the Lemma 2, 

oo oo 
k k=l k=l 

Therefore, by (5) and Lemma 2, 

P' X {B) < P>(*-\*(B))) = P X (*(B)) < P X (\J < e. 

fe 

Since e was arbitrary, this completes the proof. □ 
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Theorem 1 Suppose V is a random dynamical system with finitely many uni- 
formly continuous probability functions p e and continuous maps w e on a com- 
plete separable metric space K. Suppose that the fundamental Markov system 
associated with V has finitely many vertices, is irreducible and contractive. Then 

(i) V has a unique invariant Borel probability measure /i. 

(ii) For every x G K , 

n— 1 
k—o 

for all bounded continuous functions f. 

Proof. Apply Theorem 4 in [35] for the fundamental Markov system associated 
with V with the following justifications. In [35], the vertex sets were required 
to form an open partition of a state space in which sets of finite diameter a 
relatively compact. This was to insure that the Markov operator has the Feller 
property and an invariant Borel probability measure. Here, the Feller property is 
already given by V and the existence of invariant measures for our fundamental 
Markov system associated with T> (on complete separable space) was shown in 
[15]. Also, it was required in [35] that the probability functions p e \K i{c) shall be 
bounded away from zero, but it was only required for the proof that P x <C P y 
for all x and y in the same vertex set (Lemma 2 in [35]). The latter is given 
here by the construction of the fundamental Markov system associated with V 
and Proposition 3. 

After obtaining the result for the fundamental Markov system associated with 
V, deduce the result for V by Proposition 3. □ 

Remark 3 Note that the fundamental Markov system M 1 associated with V 
is contractive if V is contractive. 

Remark 4 Recall that there exist contractive random dynamical systems with 
strictly positive continuous probability functions which have more than one 
probability measure [8], [6]. By Theorem 1 the fundamental Markov system 
associated with such a random dynamical system can not have a single vertex 
set. 

Conjecture 1 I believe that fundamental Markov systems resolve the question 
of the necessary and sufficient condition for the stability of such random dy- 
namical systems, which has been open already for more than 70 years, in the 
following way. The random dynamical system has a unique invariant Borel prob- 
ability measure if and only if the fundamental Markov system associated with it 
is recurrent (every vertex of it is reached from any other by a finite path). Note 
that a recurrent Markov system is necessarily countable (every vertex of it can 
be coded by a finite path). 



J f d[i for P x -a.e. a E S + 
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